The security flaws that allowed this were horrendous, but the article implies that it's somehow scandalous to shut down the pipeline itself because they couldn't process billing for it. That doesn't seem scandalous to me. It's not just a question of money, it's operational management. From the article:

Colonial’s operational network uses automation systems to control and monitor the flow of fuel from refineries and tank farms into Colonial’s pipeline, and from Colonial’s pipeline into the tanks and transportation facilities belonging to suppliers and distributors.

posted by fatbird at 12:45 PM on May 20, 2021 [16 favorites]

That sounds about par for the course, vis a vis capitalism.

Also, why are we still calling it "colonial"?
posted by The Potate at 12:50 PM on May 20, 2021 [2 favorites]

Right, but the operational network was fine.

"In a statement published Saturday, it said the ransomware infected only its corporate IT network. Although the operational network that controls its pipelines and distributes fuel is separate from the corporate network and wasn’t infected, Colonial said it temporarily shut down the pipelines as a precaution to prevent the infection from spreading."
posted by Garm at 12:54 PM on May 20, 2021 [4 favorites]

If you can think of a better way of delivering critical infrastructure than mercenaries I'd like to know.
posted by adept256 at 12:54 PM on May 20, 2021 [7 favorites]

The article says it would be an accounting headache. Is it that or would they just have been giving it away for free?
Also, given that there wasn't actually a shortage aside from that caused by panic-buying, why does this matter anyway, other than as a demonstration of propaganda?
I'm all for nationalizing utilities, but in this case, what did they do that's more harmful than their business as usual?
posted by PennD at 12:57 PM on May 20, 2021 [1 favorite]

[The hackers] went on to refer Colonial to its support team, who was available via chat and could answer any questions Colonial had

Yes, that's the hackers' customer service division offering live chat with a specialist to answer questions from the owned.
posted by echo target at 1:00 PM on May 20, 2021 [28 favorites]

"The Colonial Pipeline was fine, we just shut it down for IT reasons" smacks of Harkkonen propaganda.
posted by Abehammerb Lincoln at 1:01 PM on May 20, 2021 [4 favorites]

Ok, but shutting down connected networks is also standard practice in cleanup. Even then, there's integration between operational and billing systems, and unless they've specifically planned to run the operational network without the billing system in place, it's not even necessarily going to work.

Suggesting that they could have kept the operational part going without billing (and somehow catch up later) seems extremely naive about the realities of all corporate IT systems.

Proper disaster recovery is a part of security, and colonial obviously didn't have their shit tightly wound with respect to security. Until you've anticipated and prepared for a particular scenario, "burn it all down and rebuild from the ground up" just is the proper security response.
posted by fatbird at 1:05 PM on May 20, 2021 [29 favorites]

"Now don’t get me wrong, there’s a lot to be said for gas lighting (not least the stunning fittings which survive from the wealthy middle classes)... Albert Winsor who formed the Gas Light and Coke Company, which became the largest gas company in the world. His success came despite, or maybe because of, some rather ludicrous initial claims; such as his claim that a room full of gas when entered with a candle ‘Will never inflame because it is intermixed with the air of the room’ and that gas was ‘more congenial to our lungs than vital air"

There goes the price of Diesel mitts.
posted by clavdivs at 1:05 PM on May 20, 2021 [1 favorite]

Yes, that's the hackers' customer service division offering live chat with a specialist to answer questions from the owned.

Can we hire the hackers to manage the pipeline? At least they seem competent.
posted by qxntpqbbbqxl at 1:06 PM on May 20, 2021 [6 favorites]

Imagine your boss calls you tomorrow morning and says, "Hey - we're having trouble with the time card system. If you come in today, we won't be able to record your hours like normal. It'll probably take us 2-3 months to get you paid, and when we do pay you, it'll be in the form of a bunch of $20 Visa gift cards rather than your usual direct deposit. Or, if you prefer, you can just take today off as an impromptu unpaid holiday."

Which would you choose?
posted by Hatashran at 1:07 PM on May 20, 2021 [8 favorites]

You guys realize you're not capitalists, just forced to operate within a capitalist system, right? There's no need to be a defense force for a corporation starting a panic on fuel simply because they might have lost dollars.
posted by Your Childhood Pet Rock at 1:14 PM on May 20, 2021 [38 favorites]

Someone did make an argument in this that makes sense, though. Said if you’re working IT and suddenly there’s a spike in activity and files are getting encrypted, you’re going to shut everything down as fast as possible. I don’t know if that’s indeed the case here but it makes sense.
posted by azpenguin at 1:20 PM on May 20, 2021 [2 favorites]

It's not like this doesn't happen in other businesses too.
posted by JoeZydeco at 1:24 PM on May 20, 2021

So many people I know polluted my social media feed with the same decontextualized gasoline-in-plastic-bags photo, a game of punching down that drew attention away from.... this mess.
posted by pianoblack at 1:30 PM on May 20, 2021 [3 favorites]

You guys realize you're not capitalists, just forced to operate within a capitalist system, right?

Contextualizing and shilling are different things. And why should the media get a pass for sloppiness or sensationalism?
posted by fatbird at 1:38 PM on May 20, 2021 [5 favorites]

We bail the fuckers out all the time, but when it comes to actual human suffering, they're all in on the accounting system.

Please don't pretend that the corporate profits from a critical supply line are same as a person's paycheck.
posted by Horkus at 2:04 PM on May 20, 2021 [10 favorites]

Suggesting that they could have kept the operational part going without billing (and somehow catch up later) seems extremely naive about the realities of all corporate IT systems.

This sort of thing and all sorts of other IT disasters happen in healthcare too, but we have no choice but to keep going. Of course, a lot of the downtime procedure paperwork consists of documenting things such that they can eventually be billed for, at least here in the US.
posted by TedW at 2:25 PM on May 20, 2021 [7 favorites]

HN tore this article apart a couple of days ago. Strange thing about pipelines is that they carry different types of fuels by multiplexing. Out come gasoline, then there's a plug of water, then out comes diesel, then a plug of water, and now out comes jet fuel. There's this big automated switch at the end that can route these different loads into different storage tanks.the While the 'operational' network was up (they didn't crash switches/pumps like those industrial control systems).... the logistics and accounting side of the network was down and couldn't reliably tell the big switching stuff what was coming down the pipe and where to store it. So the operational network didn't get infected, but it gets instructions from the logistics network and couldn't function while this was going on.

Sure, you could keep pumping things down the line, but you're going to end up with diesel in the gas tank, jet fuel in the diesel, etc.
posted by zengargoyle at 2:27 PM on May 20, 2021 [34 favorites]

This sort of thing and all sorts of other IT disasters happen in healthcare too, but we have no choice but to keep going. Of course, a lot of the downtime procedure paperwork consists of documenting things such that they can eventually be billed for, at least here in the US.

The (public) Irish healthcare system was just hit by a ransomware attack that is directly causing major problems for patients:

The Irish Association for Emergency Medicine urged people not to turn up at hospital emergency rooms unless they had a genuinely urgent need. The association said electronic ordering of blood tests, X-rays and scans was unavailable and clinicians had no access to previous X-rays or scan results.

Many hospital telephone systems also were not working because they are carried on computer networks, it added. The attack has also shut down the system used to pay health care workers.

I don't know if there's a single large organization anywhere where operations can just carry on as usual when the computers are down.
posted by theodolite at 2:44 PM on May 20, 2021 [4 favorites]

There was a time when fax machines and paper could manage to get the information necessary to operate the equipment. That time could be now, but it's "too hard."

I've seen people who actually give a shit about getting their product where it needs to go do exactly that when unexpected failures made it necessary. Colonial probably fired all the people who are qualified to turn the valves at the appropriate time.
posted by wierdo at 2:45 PM on May 20, 2021 [6 favorites]

What weirdo said. Wouldn’t be surprised if critical infrastructure now builds in dead-tree and landline comms contingency plans.
posted by Jubal Kessler at 2:54 PM on May 20, 2021 [2 favorites]

Yeah well there's the two arguments, right.

What if the operational system got infected as well? They will be asked why they didn't shut it down immediately. That's just standard practice. Otherwise the outage could have been even longer.

Or - say I'm a steel company. I have a contract to supply steel to many of my long standing customers. One day my billing system goes down. Obviously the right thing to do is to continue delivering steel as per my contracts, right? Then try to bill them manually later. My contracts don't get nullified due to my inability to bill.
posted by xdvesper at 2:55 PM on May 20, 2021 [3 favorites]

Also, why are we still calling it "colonial"?

Well, the majority of the pipeline runs through the areas that were once British and Spanish colonies in America, so it doesn't seem that strange. I guess someone thought it rolled off the tongue more easily than the "Most of the Eastern Seaboard and Half of the Gulf Coast Pipeline".

Personally the one I'll never understand is the "Keystone Pipeline" which is nowhere near Pennsylvania. They should sue for trademark infringement or something.
posted by Kadin2048 at 3:00 PM on May 20, 2021 [1 favorite]

You guys realize you're not capitalists, just forced to operate within a capitalist system, right? There's no need to be a defense force for a corporation starting a panic on fuel simply because they might have lost dollars.

Well a) are you sure I'm not?

And b) people have been allegedly been able to cause explosions in oil pipelines on purpose, and they seem to explode on the regular accidentally so it seems plausible and in the public interest that they not explode and be informed of the nature of the threat. Learning it was a billing issue and not a fire safety issue is somewhat relieving.
posted by pwnguin at 3:02 PM on May 20, 2021 [4 favorites]

Certainly the reason this pisses people off is that it double-counts Colonial's failure. One, they didn't sufficiently backup and protect their computers. OK, fine, I guess shit happens, and there is an operational argument to be made for shutting everything off while you figure out what's what.

But it also set off a secondary problem (panic buying and a fuel shortage) that might not have occurred if they had made it public that the gas delivery system was functional, rather than try to make it seem like those dastardly hackers are choking off the fuel supply. This of course had the tidy effect of making the hit to Colonial's bottom line smaller (versus delivering fuel without proper billing) at whatever the externalized cost of fuel shortages accrued to the public.
posted by axiom at 3:05 PM on May 20, 2021 [2 favorites]

There was a time when fax machines and paper could manage to get the information necessary to operate the equipment. That time could be now, but it's "too hard."

I've seen people who actually give a shit about getting their product where it needs to go do exactly that when unexpected failures made it necessary. Colonial probably fired all the people who are qualified to turn the valves at the appropriate time.


This is so ridiculous and wrong I don't even know where to start. If your systems have been completely computerized and automated for years you cannot just switch to paper and faxes and manually turning valves at the drop of a hat. Like, all of your background customer information that you'd need to get your paper processing off the ground is on the computer. All of the information about who is getting what and when is on the computer. The valves that you think people should be turning may not even be easily accessible to manual control. I could keep going.
posted by Anonymous at 3:16 PM on May 20, 2021

What if the operational system got infected as well? They will be asked why they didn't shut it down immediately. That's just standard practice. Otherwise the outage could have been even longer.

Or more dangerous. Remember the Ukraine power utility hack from a few years back? Or how about the recent attempt to contaminate a towns water supply?

It's not uncommon for these groups to have a presence on the network for a while before pulling the trigger on encryption. The source of the attack would be quickly identifiable as Russian where the difference between criminal and state actors it murky at best. How would people be reacting if there was some sort of spill and they hadn't shut things down? If I were an admin on this network I would have wanted to make damn sure the ICS side was clean before firing things back up.
posted by calamari kid at 3:30 PM on May 20, 2021 [3 favorites]

the gas delivery system was functional,

But it wasn't: it's directed by the logistics and accounting systems, which were down. You don't just start using stickies to keep track of things and twiddling knobs.

Certainly the reason this pisses people off is that it double-counts Colonial's failure.

I'm not pissed off, but I'm annoyed that, in the Age of Disinformation in which we now live, I'm getting pushback on simply trying to have an accurate appraisal of the situation. I work in IT. I just went through a comparable security event at a Fortune 100 who's a client of my employer. I'm aware in my bones just how much daily operations in a significantly-sized company are hard to get right even on good days, and how painfully hard it can be to get beyond the daily operations and actually start planning for disaster recovery.

None of this is to excuse Colonial for its failures. All I want, all I try to do each day, is just feel like I actually know what's going on and I'm not just being swept along by a consensus. Sorry if that gets in the way of someone else's dudgeon.
posted by fatbird at 3:30 PM on May 20, 2021 [26 favorites]

If your systems have been completely computerized and automated for years you cannot just switch to paper and faxes

Ireland's health services were attacked by ransomware. They are refusing to pay and are switching to managing things via paperwork, until systems can be cleaned and be brought back online. Running things won't be easy, in the meantime, but it isn't going to be impossible, either. Colonial could have done the same, but will not. That's a positive choice.
posted by They sucked his brains out! at 3:31 PM on May 20, 2021 [3 favorites]

Look, if there’s anything we can all agree on, it’s that we’re smarter than the people who do this for a living.
posted by Huffy Puffy at 3:50 PM on May 20, 2021 [38 favorites]

I've seen people who actually give a shit about getting their product where it needs to go do exactly that when unexpected failures made it necessary. Colonial probably fired all the people who are qualified to turn the valves at the appropriate time.

Imagine if you no longer had access to your smartphone/laptop. How sure are you that you could contact your own family? I can tell you right now that, while I have my wife's phone number memorized, as well as my own, I couldn't begin to tell you what my kids' numbers are.

How in heaven's name are you going to ensure that you're getting the right products to the right places without computers? It is not reasonable to expect any business to keep a paper backup of all of their records available at the drop of a hat. If the problem is that this business is integral to the country running, then the government should either ensure via laws that they pass muster on a certain level of security, or nationalize the industry.
posted by nushustu at 4:11 PM on May 20, 2021 [3 favorites]

The valves that you think people should be turning may not even be easily accessible to manual control. I could keep going.

The Chemical Safety Board would have unkind words with anyone who chose to design a system such that the valves could not be quickly operated manually in the event of a systems failure.

The way a pipeline works means that the information about who is receiving what product when is not contained solely within the pipeline operator's system. Customers have to know when the product will arrive, most obviously so that they can plan in advance what tank to put the stuff in. It's absolutely possible to get all the information needed to continue deliveries. Hard, yes, but also possible, with some delay.

The lack of resilience is a choice, not some inevitable law of nature.
posted by wierdo at 4:21 PM on May 20, 2021 [5 favorites]

But it wasn't: it's directed by the logistics and accounting systems, which were down.

OK, but it's not a commandment from on high that your accounting system should be so tightly integrated with your pipeline that one dying necessarily brings down the other. What I'm saying is that if the people administrating this system (1) intertwined those two services and (2) had no contingency plan for network security events like ransomware and (3) think SINGLE POINT OF FAILURE is a a great cost-saving or efficiency mechanism, then maybe they shouldn't be allowed anywhere near the decision making apparatus that governs the pipes in your kitchen, let alone those that control gas delivery for a big hunk of the country. These people are bad at their jobs, and pushing a narrative of "the hackers did it, we're blameless angels" isn't flying.

Look, if there’s anything we can all agree on, it’s that we’re smarter than the people who do this for a living.

Hi, I'm trained (M.Sc.) in these specific problems (not gas delivery, but systems architecture and threat modeling). One reason the company I work for is reasonably resistant to ransomware, as one type of threat that's germane here, in part is because we have multiple redundant systems and distributed backups, and a plan for what to do in the event shit hits the fan to try to at least recover to a 'working but limping' state as quickly as possible. This isn't some huge feat of engineering, it's common best practices. They created a system that does not have the basic ability to fail gracefully and instead exploded (figuratively) when faced with a fairly low-rent ransomware attack. They ain't coming out of this smelling like roses.
posted by axiom at 4:27 PM on May 20, 2021 [11 favorites]

These people are bad at their jobs, and pushing a narrative of "the hackers did it, we're blameless angels" isn't flying.

To be clear I was saying THEY are the ones pushing that narrative, not anyone here.
posted by axiom at 4:28 PM on May 20, 2021

I haven't seen anyone, let alone in this thread, say that this wasn't a massive failure of preparation and security on Colonial's part.

Colonial could have done the same, but will not. That's a positive choice.

Their pipeline was down for five days. Could they have figured out and implemented a manual operation of the system in less time than that?
posted by fatbird at 4:43 PM on May 20, 2021 [3 favorites]

Also, why are we still calling it "colonial"?

The frakkin' Cylons shut down oil distribution all across the Eastern seaboard! The only capacity we've got left is a rusted-out old pipeline whose computers were never networked...
posted by justsomebodythatyouusedtoknow at 4:53 PM on May 20, 2021 [9 favorites]

OK, but it's not a commandment from on high that your accounting system should be so tightly integrated with your pipeline that one dying necessarily brings down the other.

Actually, it is. Pipelines are regulated entities, and billing is thus tightly coupled to operations as a result of those regulations. Shutting down the pipeline was exactly what they're required to do.
posted by Runes at 6:06 PM on May 20, 2021 [5 favorites]

Their pipeline was down for five days. Could they have figured out and implemented a manual operation of the system in less time than that?

This particular organization? I have no idea, and am not particularly interested in finding out. Why is this the benchmark? Surely it should be something like "could a reasonably competent organization with the simple foresight to plan for contingencies like ransomware have ameliorated this attack faster than the five days it took this bunch of hacks?" I suspect the answer to THAT question is "yes." To analogize, if you show up on the day of the test without having studied, shouldn't the teacher still grade your exam on a curve with the whole class?
posted by axiom at 6:08 PM on May 20, 2021 [1 favorite]

Actually, it is. Pipelines are regulated entities, and billing is thus tightly coupled to operations as a result of those regulations. Shutting down the pipeline was exactly what they're required to do.

Genuinely curious, why is the regulation set up this way? I'm not a lawyer so I have no idea what the relevant laws would be, but from a first-principles approach to making systems like this, I would love to know how it came to be that a legislature saw fit to mandate coupling gas delivery with billing. Am perfectly happy to direct some of my ire at bad laws as well, lord knows there's no shortage of those.
posted by axiom at 6:16 PM on May 20, 2021 [1 favorite]

Surely it should be something like "could a reasonably competent organization with the simple foresight to plan for contingencies like ransomware have ameliorated this attack faster than the five days it took this bunch of hacks?" I suspect the answer to THAT question is "yes."

I suspect you have no idea of the complexities involved.

Genuinely curious, why is the regulation set up this way?

Pipelines are natural monopolies.
posted by Runes at 6:24 PM on May 20, 2021 [6 favorites]

Pipelines are regulated entities, and billing is thus tightly coupled to operations as a result of those regulations. Shutting down the pipeline was exactly what they're required to do.

That effectively makes it part and parcel of the operations system, and as such, security of the billing system was a board-level responsibility that was repeatedly dismissed.
posted by wolpfack at 6:25 PM on May 20, 2021 [2 favorites]

Why is this the benchmark?

Because what some here are doing, including the article that started all of this, is blaming colonial for shutting down the pipeline rather than somehow manually operating it, making them responsible for the panic buying that caused so much hardship. So imagine you're colonial, and have to decide what to do now that you've turned off the billing system and the operations system because of ransomware. Do you say "the number one priority for everyone here is to get the systems up and running again" or do you say "Ok, everyone, we need to figure out how to run this manually while IT reformats the computers"? And you have to judge which is more quickly going to get you back to operational status. With an outage of five days, I'd say the former.

I can also speak to the regulation aspect because I've worked briefly on the systems governing the management of transfer of fluids like the pipeline carried. I can't speak to the regulations themselves, but the systems that managed the transfer were required to maintain an auditable trail of data from sensors that allowed measuring volumes in the millions of gallons to a precision of fractions of a gallon. You don't get that by having someone standing by with a walkie-talkie, waiting for someone to send "Alright, Cletus, close the valve NOW NOW NOW!"

I feel like I'm in a battle here against the forces of "I don't understand why this takes so long, why don't you do it this way?"
posted by fatbird at 6:34 PM on May 20, 2021 [22 favorites]

I suspect you have no idea of the complexities involved.

I'm sure that there are lots of things about the pipeline industry I don't know, it's not my field. But are you trying to insinuate that the expectation should be that they all fail this test? If so, that's terrifying, and not without precedent. Some other folks mentioned hospitals upthread, and the general weaknesses of many hospital IT systems should keep us all up at night (and yes, I think the same thing there; it's possible to setup and maintain a more secure standard at hospitals than is often done).

Or do other pipeline companies successfully plan for and ameliorate these types of problems? Because if it's that, than my not knowing of the specific complexities is kind of irrelevant, since clearly the people who are responsible for these systems have solved this problem before.

Pipelines are natural monopolies.

OK, sure, but that does not seem to also imply "therefore the billing system and the delivery systems must be wholly dependent with no redundancies."
posted by axiom at 6:36 PM on May 20, 2021 [2 favorites]

They’re pumping a million barrels of gasoline a day across the South. I’m glad they didn’t try to wing it.
posted by Huffy Puffy at 6:39 PM on May 20, 2021 [2 favorites]

Hello, I help people recover from ransomware attacks, professionally. Some thoughts:

1. The failure to have adequate backups means these people are laughably incompetent. It's very common, but that doesn't make it less incompetent.

2. They operate a fuel pipeline. They are critical infrastructure. That they don't have a business continuity plan for how to minimise service disruption in the event of a cyberattack is also laughably bad. Most people in this position would shut down all systems because they don't know in the moment what's been compromised, but you can't do that willy-nilly for systems that people rely on. Or, at least, it shouldn't be your go to move. It's possible that they had no other option, but given the nature of the service they really should have had backup systems. Having said that, event the most sophisticated orgs have system outages; sometimes continuity plans fail.

3. Ransomware gangs pick soft targets. These guys must have had a pathetic perimeter.

tl;dr, it's almost certain that the pipeline operators were a total clownshow, from a security and systems perspective.
posted by His thoughts were red thoughts at 6:41 PM on May 20, 2021 [20 favorites]

So imagine you're colonial, and have to decide what to do now that you've turned off the billing system and the operations system because of ransomware.

I'm imagining I'm Colonial and asking a different question: what did I do in the weeks, months, and years before the ransomware attack to plan for this sort of eventuality, and to allow my systems to recover from failures (or better yet, avoid failing in the first place)?

I'm glad they didn't try to wing it.

Me too! What I want to know is, how did they get themselves into this position in the first place, not what did they do to the barn door once the cows had fled.
posted by axiom at 6:44 PM on May 20, 2021 [3 favorites]

Pipelines are natural monopolies.

US businesses have a just-in-time approach to inventory. One pipeline supplies inventory to, basically, everybody in a geographic area. Stuff like this is why good government is important, and why running the country like a business doesn't work well. If gasoline for consumers, delivery fleets, fuel for home heat, etc., is critical, there should be a better plan. Add panic buying and it was a mess.

The Trump Administration had a horrible track record on CyberSecurity. I worry that we don't really know how bad network security really is, how vulnerable we may actually be.
posted by theora55 at 6:48 PM on May 20, 2021 [3 favorites]

I have no idea where the no backups/no redundancy claim is coming from, I haven't seen it reported anywhere. I'd be very surprised indeed if that was the case.
posted by Runes at 6:57 PM on May 20, 2021 [1 favorite]

There usually is some buffer in petroleum delivery thanks to those big storage tanks you see at terminals. A wave of panic buying can use up a couple weeks worth of supply in a couple of days, though.

I say usually because there are times near the seasonal change in gasoline formulation where those tanks get as close to zero as the companies involved can manage.
posted by wierdo at 6:57 PM on May 20, 2021 [1 favorite]

I'm imagining I'm Colonial and asking a different question: what did I do in the weeks, months, and years before the ransomware attack to plan for this sort of eventuality, and to allow my systems to recover from failures (or better yet, avoid failing in the first place)?

The absolute worst thing to do in a crisis is try to carry out a post-mortem then and there. No one has excused colonial's preparedness prior to the attack, which was terrible, and in a just world would result in the C suite going to jail and the critical infrastructure nationalized until or if a competent private entity can be found.

Or, to paraphrase Ronnie Chen:

Axiom, now is not the time for negative energy. We are in problem solving mode. There's plenty of time at a later date to assign blame to all parties responsible for this fucking fiasco

posted by fatbird at 7:06 PM on May 20, 2021 [2 favorites]

I have no idea where the no backups/no redundancy claim is coming from

I am sure more details will come out later, but if you have the capability to restore from backups, you don't usually pay the ransom, unless there's another reason (for example, restoring is somehow more expensive, or the data at risk of being publicly revealed contains some other skeletons from your closet, at which point my question is dear god what might those be). With redundant systems you don't shut down your pipeline for 5 days, unless your idea of what a redundant system is there for is wildly different from mine.

The absolute worst thing to do in a crisis is try to carry out a post-mortem then and there.

I'm not in a crisis, and neither are you. We don't work for Colonial, we're just two people on a message board. And from way the hell over here, it seems to me the real meat of the issue is 'why is our infrastructure susceptible to ransomware gangs, how deep does the rot go, and what does it look like when competent people are in charge of that infrastructure' not 'what should this one group of bozos be doing-or-not, right now.'
posted by axiom at 7:20 PM on May 20, 2021 [5 favorites]

What's being argued is whether their actions in leaving the pipeline down for five days were appropriate or exacerbated the crisis. No one is arguing that their actions leading up to the crisis were appropriate or reasonably sufficient. They obviously weren't.
posted by fatbird at 7:27 PM on May 20, 2021 [1 favorite]

I would love to know how it came to be that a legislature saw fit to mandate coupling gas delivery with billing.

Let me introduce you to your new best friend the Federal Register. Congress has vanishingly little to do with industry regulation.
posted by pwnguin at 8:41 PM on May 20, 2021

It's frustrating that Capitalism rewards Colonial for being incompetent, by jacking up prices. Why would they have security?

Just like the Comstock company during the Texas Freeze. There s a lot of money to be made in providing a shittier service.

The criminals, having been borne outside of this system, apologized, because they assumed that the united states was enough of an entity to secure its gas supply.

After all, these pipelines are put through people's homes for the sake of the nation, right? We gave these companies the right to expropriate private property. Surely we asked that these companies competently operate in return, right?
posted by eustatic at 9:17 PM on May 20, 2021 [3 favorites]

They’re pumping a million barrels of gasoline a day across the South. I’m glad they didn’t try to wing it.
posted by Huffy Puffy

What in the world makes you think that they are not winging it?
posted by eustatic at 9:21 PM on May 20, 2021 [4 favorites]

By that I mean, Colonial has accidents and explosions all the time. There was a large one on May 10th that isn't public yet, but a whole town was quaking in fear at the fire across the night sky.

Look them up in the National Response Center data Sometime.
posted by eustatic at 9:27 PM on May 20, 2021 [5 favorites]

They’re pumping a million barrels of gasoline a day across the South. I’m glad they didn’t try to wing it.

Wing it, like when they leaked 1.2M gallons of gasoline in a nature preserve, last year?
posted by They sucked his brains out! at 9:29 PM on May 20, 2021 [6 favorites]

This is an interesting discussion, but here's one thing which really stuck out to me:

> Strange thing about pipelines is that they carry different types of fuels by multiplexing. Out comes gasoline, then there's a plug of water, then out comes diesel, then a plug of water, and now out comes jet fuel.

I would love to learn more about this.
posted by The Lurkers Support Me in Email at 9:30 PM on May 20, 2021 [5 favorites]

Pipeline batching is the search term that may help.
posted by mireille at 10:00 PM on May 20, 2021 [4 favorites]

It's frustrating that Capitalism rewards Colonial for being incompetent, by jacking up prices. Why would they have security?

I wasn't paying close attention -- is there some mechanism by which a pipeline can profit by shipping gasoline at a high price caused by them simultaneously not shipping gasoline? Seems like a self-defeating proposition.
posted by pwnguin at 10:34 PM on May 20, 2021 [1 favorite]

I suspect you have no idea of the complexities involved

And to be fair, they did try turning it off and on again.
posted by flabdablet at 4:15 AM on May 21, 2021 [4 favorites]

Funny thing about IT systems, and software in general: The people who build these things (including me) almost always underestimate and thus fail to appropriately model the complexity of the problems their system handles. But at the same time, many people also simultaneously believe that the complexity of the systems are such that they can't be changed or replaced. Of course both of these things can be true at the same time: It just means that the system is complex in the wrong ways.

In any case, for critical infrastructure, they should have been able to fall back to manual control. Nothing about measuring millions of gallons accurately or routing different fuels precludes this as a possibility. Obviously they were not set up to do so. It's fair to point this out as a problem with their system.
posted by Nothing at 4:50 AM on May 21, 2021

In any case, for critical infrastructure, they should have been able to fall back to manual control.

Is it even possible to fall back to manual control and maintain the speed at which the economy is accustomed to?
posted by pwnguin at 9:29 AM on May 21, 2021 [1 favorite]

It's probably faster than leaving things in the Off position.
posted by They sucked his brains out! at 9:34 AM on May 21, 2021